Optimal hyper-scalable load balancing with a strict queue limit

Load balancing plays a critical role in efficiently dispatching jobs parallel-server systems such as cloud networks and data centers. A fundamental challenge the design of load algorithms is to achieve an optimal trade-off between delay performance implementation overhead (e.g. communication or memory usage). This has primarily been studied so far from angle amount required asymptotically performance, particularly vanishing large-scale systems. In contrast, present paper, we focus on arbitrarily sparse budget, possibly well below minimum requirement for delay, referred hyper-scalable operating region. Furthermore, may only be admitted when specific limit queue position job can guaranteed. The centerpiece our analysis universal upper bound achievable throughput any dispatcher-driven algorithm given budget limit. We also propose scheme which operate at message rate enforce limit, while allowing server states captured via closed product-form network, servers act customers traversing various nodes. distribution leveraged prove that tight proposed throughput-optimal many-server regime constraints. Extensive simulation experiments are conducted illustrate results.